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Abstract 

The magnitude of correlations between stimulus-driven responses of pairs of neurons can 
itself be stimulus-dependent. We examine how this dependence impacts the information carried 
by neural populations about the stimuli that drive them. Stimulus-dependent changes in corre- 
lations can both carry information directly and modulate the information separately carried by 
the firing rates and variances. We use Fisher information to quantify these effects and show that, 
although stimulus dependent correlations often carry little information directly, their modula- 
tory effects on the overall information can be large. In particular, if the stimulus-dependence is 
such that correlations increase with stimulus-induced firing rates, this can significantly enhance 
the information of the population when the structure of correlations is determined solely by 
the stimulus. However, in the presence of additional strong spatial decay of correlations, such 
stimulus-dependence may have a negative impact. Opposite relationships hold when correlations 
decrease with firing rates. 

1 Introduction 

The impact of correlations on information encoded in neural tissue is a subject with a substantial 
history. We start our discussion with [49], which reported significant correlations between neuronal 
responses in paired recordings of neurons in the visual area of monkeys. Correlations were deemed 
undesirable, as they lead to a decrease in the signal-to-noise ratio of the summed population ac- 
tivity |241 [9] . Despite this impact on the signal-to-noise ratio, correlations in the neural response 
can increase the information that a population of neurons carries about a signal pQ. The impact 
of correlations on coding depends in a complex way on their distribution over the neuronal popu- 
lation [33 US EH ED El HQl [271 HI [13 SH [S]. As the range of potential patterns of correlation is 
vast, and has not been characterized in most neurobiological systems, the effect of correlations is 
not fully understood. 

In many studies to date, the correlation coefficient between the responses of pairs of neurons 
was assumed to be independent of the stimulus driving the response. In particular, it was assumed 
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that covariances between cell responses change together with the variance so that the correlation 
coefficient remained constant. Information about stimulus identity could then be encoded solely in 
the rate and variability of single cell responses [US31114, 46 . However, experimental findings suggest 
that correlations themselves vary with stimuli [181 133 0211 EH EH [T71 [11] . More specifically, it has 
been shown in that correlations in the visual cortex (VI) vary with the stimulus orientation and 
contrast. In [7], it was demonstrated experimentally that in certain situations changes in perceived 
brightness are related to changes in neural correlations. Responses to prey-like vs. conspecific-like 
stimuli in electric fish have also been demonstrated to evoke responses with different correlation 
structure [TT] . 

Here, we concentrate on a particular form of stimulus-dependence, in which correlations depend 
on stimulus-evoked firing rates (although many of our formulas hold more generally). In recent work, 
we have shown that spike-to-spike correlations due to common inputs increase with firing rate for 
neural models and in vitro neurons [T7J . This effect was observed in vivo in the anesthetized visual 
cortex [26, 22J and, in certain experimental regimes, for motoneurons in vitro [8]. In the oculomotor 
neural integrator the opposite effect was observed: correlations decreased with rate [2], perhaps due 
to recurrent network interactions. We will study both of these cases, illustrating strongly differing 
effects of stimulus-dependence in each. 

The goal of this paper is to examine, from a theoretical perspective, the impact of stimulus- 
dependent correlations on population coding. Previously, changes in discriminability due to changes 
in the covariance matrix of pairs of cells and small (3-8 cell) ensembles were examined by [6]. Also, 
a series expansion of mutual information to isolate and quantify the effects of stimulus-dependent 
correlations has been developed [32J. Similarly, [30J use mutual information to assess the impact 
of tuned correlations measured in primate VI. We take a somewhat different approach based on 
computing the impact of the stimulus dependence of correlations on the Fisher information (Ip) 
for populations of neurons whose response is described by tuning curves [I]. 

There are at least two distinct ways in which the stimulus dependence of correlations can impact 
Fisher information. First, the fact that patterns of correlation across a population are adjusted as 
stimuli change can have a strong "modulatory" impact on the information that other features of 
the neural response - such as firing rates - carry about the stimulus [301 123] . We refer to this effect 
as correlation shaping. To better understand this, note that a stimulus-independent correlation 
structure may be optimized for one stimulus. However, stimulus-dependence offers the possibility 
that the correlation structure is adjusted, and optimized, for a range of stimuli [3]. In a related 
effect, adaptation has been shown to modify correlation structure and increase Ip \27\ 123]. 

Secondly, information may be encoded directly by changes in the level of correlation between 
neurons, in addition to encoding via changes in firing rate and variance. We refer to this mechanism 
as correlation coding. One scenario where correlation coding clearly dominates if stimuli only affect 
the correlation structure, leaving rates and variances relatively constant, as has been observed 
experimentally [171 El E] • 

The balance of the paper proceeds as follows. We start by defining our statistical description of 
the neural response to stimuli in Section [2l The information in the response of two cells is studied in 
Section [3l As we show, the insights gained from this case can be extended to small populations, but 
do not always apply to larger populations. In Section [4] we study the information in the response 
of a large population. Here, we find that correlation shaping effects can be substantial, and often 
dominate over correlation coding. In Section [5] we extend the model to address additional structure 
of correlations across the population, by including decay of correlations that depends explicitly on 
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the spatial or "functional" distance between preferred stimuli of neurons, as shown experimentally. 
We find that the impact of correlation shaping in the presence of such a decay continues to be 
strong, but that correlation coding also plays a significant role. We conclude with a discussion of 
the results. A number of analytical results used in the main body of the paper, which may be of 
independent interest, are derived in the appendices. 

2 Setup 

Structure of correlations We consider a population of N neurons responding to a stimulus 
described by a scalar variable 9 (for example, the orientation of a visual grating). The number of 
spikes fired by neuron i in response to stimulus 9 during a fixed time interval is given by 

r i (9) = f i (e)+r H (9), (1) 

where fi{9) is the mean response of neuron i across trials, and rji(9) models the trial-to-trial 
variability of the response. We use boldface notation for vectors, so that r(9) denotes the mul- 
tivariate random variable r{9) = [r\(9), r2(9), . . . ,rN(9)] T . For simplicity, we sometimes suppress 
dependences on 9. 

We assume that rj follows a multivariate distribution with zero mean and covariance matrix 
Q(0) defined by 

Qi ;j (9) = S iijVi (9) + (1 - S i j)p iJ (9)^v i (9)v j (9). (2) 

Here Vi(9) is the variance of the response of cell i, and — 1 < Pij(9) < 1 is the correlation coefficient 
of the response of cells i and j. Although most of our results will be discussed in the range of 
small to intermediate correlations, pij < 0.5, a similar analysis can be used to study the behavior 
of populations close to perfect correlations, pi ~ ~ 1. Assumptions on the form of the distribution, 
beyond this covariance, are made only where needed. 

For studies of stimulus-dependent correlations in small-to- intermediate populations (Section [3]), 
we will allow for general forms of pij(9). When we study large populations (Sections 0] and [5]), we 
will assume that 

ftj (fl) = 5y(9)c(^-^), (3) 

where (pi and <j)j are the preferred stimuli of neurons i and j respectively. The stimulus independent 
term c(4>i — (fij) represents the spatial or functional structure of correlations in the population. It 
describes how correlations vary across the population according to their preferred stimuli, perhaps 
due to hardwired differences in the level of shared inputs. For instance, neurons which prefer similar 
stimuli are frequently closeby in the cortex, and may share a larger number of common inputs than 
neurons that exhibit different preferences [491 128j. Moreover, the set of neurons upstream of two 
cells with similar stimulus preferences may also undergo common fluctuations in their activity. 
Therefore, c{<pi — 4>j) = c (A</>) is frequently assumed to decrease with the functional distance A(p. 
We will refer to this simply as "spatial decay" [161 113 HE] • 

We emphasize that it is the stimulus dependence of the correlation coefficient, pij(9), that 
distinguishes the present work from several previous investigations [U HBJ 03] . This dependence 
enters through the term Sij(9) \26\ \T7\ I22j . We mainly investigate cases in which correlations 
between pairs of cells increase, decrease, or have a single maximum with respect to the evoked 
firing rates /j and fj [T71 HSJ [261 S 12] ■ However, our results could also be applied to cases with 
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other relations between Pij(9), fi, fj, Vi, and Vj such as those arising for different circuit and 
nonlinear spike generation mechanisms (cf. Fig. 4 of |17j). 

For large populations, we extend the multiplicative model in [42J to the case of stimulus- 
dependent correlations by assuming that 

Sij(6) = Si (6) Sj (8) , (4) 

where —1 < Si(8), Sj(9) < 1. Here Sj(0) may be thought of as the propensity of a neuron's response 
to be correlated, and sf{6) as the correlation between two neurons which respond equivalently to 
the stimulus. There are several reasons for adopting the form given in Eq. @. Firstly, this form 
of pij arises for small to intermediate correlation in neuron models producing a spike train with 
renewal statistics \17\ 145] . Moreover, in this case correlation has also been shown to vary with 
the geometric mean of the firing rate of pairs of cells in vivo [26, 17], which can be modeled using 
Eq. Additionally, this form keeps the computations at hand analytically tractable for large 
population sizes, and limits the number of cases under study. 



Fisher information To quantify the fidelity with which a neuronal population represents a 
signal, we use Fisher information |4 H I16j. For the probability distribution p[r|0] of the spike count 
vector r given stimulus 9, the Fisher information is defined as 

I F (9) = (-^logp[r\9] 

where < • > denotes expectation over the responses r. The inverse of the Fisher information, 
1/If(9), provides a lower bound on the variance (i.e., an upper bound on the accuracy) of an 
unbiased decoding estimate of 9 from the population response [14|, 116] . Fisher information is 
directly related to the discriminability d' between two stimuli 9 and 9 + A9, since d' « A9^/IfJ9) 
for small A6> 

The Fisher information can be written as [25] 

I F = I— + If\ (5) 

Here 

I— = f^Q-if (6) 

is known as the "linear approximation" to the Fisher information, or the linear Fisher information. 
Specifically, the inverse of f^Q-if gives the asymptotic^ error of the optimal linear estimator 
of the stimulus, for a response to the stimulus that follows any response distribution that has 
mean f(6*) and covariance Q(9) [351 13 HO]- In particular, this applies to gaussian or nongaussian 
distributions. 

The second term, i|? v , does depend on the the form of the response distribution, beyond its 
covariance. In the following, whenever computing /™ v , we assume that r\ follows a multivariate 



1 Here, asymptotic implies that the optimal linear estimator is constructed based on full knowledge of the mean 
and covariance of the underlying stimulus-response distributions. 
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Gaussian distribution, so that [25] 

IJ? V = iTr [Q'Q^Q'Q 1 ] . (7) 

As we explain below, correlation coding affects only I|? v , while correlation shaping affects both 
If? ean and If v . 



3 The cases of cell pairs and small populations 

We start by considering the impact of correlations on the information carried by cell pairs, and 
small populations (N < 1/p). This was the setting of many experimental studies which addressed 
the role of correlations in the neural code [331 HI EH GH1 [Ml [23] . We use analytical expressions 
to show that, depending on correlation structure, correlation shaping can have either a positive 
or negative impact on /™ ean . Most beneficial are high correlations between neurons with different 
stimulus preferences, and low correlations between neurons with similar preferences. For small to 
intermediate correlations, /™ ean « l Fy and hence correlation coding has little effect. These results 
are in agreement with previous observations [361 El E] • We emphasize that these results can be 
expected to hold only when N < 1/p: in subsequent sections we show that the intuition gained 
from studying cell pairs may not always extend to larger populations. 



Fisher information in cell pairs We first consider two cells whose response follows a bivariate 
Gaussian distribution given by Eqs. ([THS]) For two neurons, we write the correlation coefficient as 
Pi, 2 = P2,i = Pi and obtain 
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(8) 

where all derivatives are taken with respect to the stimulus 9. 

Intuitively, /™ ean and /|? v represent the contribution of changes in the firing rate and covariance, 
respectively, to the Fisher information. While /™ ean has been studied previously, I|? v has only been 
examined for stimulus independent correlation coefficients, i.e. when p' = [i~l 146 ^ 14^ 1 142] . We 
separate the influence of stimulus dependent changes in correlation on Ip as follows: 



Correlation Coding. The last of the five terms in the sum ([8|) is only present when p' ^ 0, and 
captures the amount of information directly due to changes in correlations |47[ [TT] . We refer 
to terms in Ip that are nonzero only when p' ^ as the contribution of correlation coding. 
If /{ = = v'± = v' 2 = then all information is due to correlation coding. It is necessary to 
use a nonlinear readout (decoding) scheme to recover this information 
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• Correlation Shaping. /™ ean is affected significantly by the level of correlation, p. As (7™ can ) _1 
measures the error in the optimal linear estimate of the stimulus, the impact of changes 
in correlation structure on /™ ean represents the amount by which correlations shape the 
information available from linear readouts of the response. We refer to this effect as correlation 
shaping. 

This terminology anticipates the discussion of larger populations, where we will be interested 
in how the spatial structure together with stimulus dependent changes of pij affect If- We note 
that stimulus-dependent correlations can also impact the information available from the variance 
of the neural response (see the third term in Eq. ©). This is another form of correlation shaping 
with a marginal impact in the cases we discuss. 

We first examine the effect of correlation shaping. A number of previous studies concluded that 
an increase in correlation, p, can positively impact J™ ean for pairs of neurons that have different 
"normalized" mean responses to the stimulus (f[/y/vi fUy/^)- The effect tends to be negative 
if the responses are similar (f[/y/vi ~ f 2/^/^2)- Intuitively, correlations can be used to remove 
uncertainty from noisy responses of neuron pairs with differing response characteristics [3 1 ^ [3], fT], 146] . 

Indeed, the first term in Eq. [fi/^/^i - fUy/vi] /(I 

— p 2 ), increases with p, unless f[/y/vi = 
f'il\fv2- The resulting increase in discriminability is illustrated in Fig. [1] where we show the bi- 
vartiate distribution pirx^r^) of the response to two nearby stimuli 6a and Ob- In panels a) and 
b), v[ = v' 2 = p' = f'2 = 0, but f[ 7^ 0, so that only the first term in j~™ ean contributes to If- 
In this example, an increase in correlation leads to a large increase in If- In Fig. Q]this increase 
results in improved discriminability between the stimuli, i.e. a reduction of the probability that the 
two stimuli will lead to the same response. However, when the two neurons respond similarly to 
the stimulus, f[/y/v{ ~ f 2/^/^2, the second term, 2 [f{f 2/ -\/ v l v 2\ /(I + p), dominates. An increase 
in correlations leads to a decrease in /™ ean |46[ [3] which is reflected in decreased discriminability 
between the stimuli (See panels c) and d) of Fig. [1]). High values of the correlation coefficients have 
been used in Fig. [1] for easier visualization. 

In contrast, correlation coding typically has a small effect in the case of two neurons, as the 
term /|? v is far smaller than /™ ean . There are two reasons for this: The first holds only in the small 
correlation regime. Note that p enters Jf? v at 0(p 2 ), while it enters i|? ean at O(p). The second 
holds for a larger range of correlation strengths: v\jvi and p' are typically far smaller than fl/y/vi 
and, as a resullH, /|? v <C /™ ean . Therefore, under fairly general assumptions, the dominant effect 
of correlations on Fisher information for cell pairs is via correlation shaping of I™ ean . 

Only close to perfect correlation, where p ~ 1, is the impact of correlation coding potentially 
significant. Assuming that p' = 0(1) as p approaches 1, and letting e = 1 — p 2 , we have Ip = 
2e~ 2 (p') 2 + 0(e _1 ). Therefore, when p is close to 1, most information about a stimulus can be 
carried by correlation changes. The balance between /™ ean and /|P rr close to perfect correlations 
strongly depends on the behavior of p' as p approaches 1. If p' approaches as p approaches 1, as 
in [T7], /™ ean may continue to dominate. 

2 In detail: if responses are given by counting spikes over ~ 1 second, then typically / takes values substantially 
greater than 1. If firing is Poisson-like, then v ~ /. This leads to the stated dominance of f'/y/v among these terms. 
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Figure 1: Illustration of correlation shaping for neuron pairs. Each panel shows 50% level curves 
of the joint density p(r±, r2) in response to two nearby stimuli 9 a (dashed line) and 9b (solid line). 
In all cases, v\ = i>2 = 1. A change from stimulus 9a to 9b is assumed to affect only the fi, so that 
Ip™ = 0. The beneficial effect of correlations on /™ ean (first term in Eq. ([8])) is illustrated in panels 
a) and b). Here /[(9a) 7^ /^a), an d increased correlations improve discriminability. In contrast, 
/[(9a) = /^a) i n panels c) and d), and increased correlations reduce discriminability. In panels 
a) and b): /i(9a) = H(6a) = 1> while /i(9b) = ^,h(9s) = 1-1. In panel a), p = 0.2, while in 
panel b), p = 0.99. In panels c) and d) /i(6a) = /2(9a) = 1, and /i(9b) = h(^B) = l l- In panel 
c), p = 0.1, and in panel d), p = 0.99. 
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Fisher information in small populations Many of these observations extend to small popu- 
lations of neurons with low correlations. Let 
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Here (If)™ 63,11 and (If)] 31 are 0(1), while (If)™™ is O(pfj). Therefore, If is a sum of contributions 
from individual neuron responses ((If)^^ and (If)] 81 ) and corrections of higher order in p due 
to correlations in the response. 

Only the term — j v£j flfjPij/ \/ v i v i m /™ ean is of first order in p. This term therefore 
dominates the correction when correlations are small to intermediate. In this case, correlations 
between differently tuned neurons again increase Ip, and those between similarly tuned neurons 
decrease Ip- If correlations pi , across the (small) population are stronger between neurons i and j 
for which ft and /j have opposite signs and weaker when these signs are the same, they increase Ip. 
This is in agreement with the two cell case discussed above, as well as previous results [6j [371 El US] • 

Eq. (jHJ) is general, under the assumption that the response follows a multivariate Gaussian distri- 
bution. However, the approximation starts breaking down when N exceeds 1/pij (See Appendix [Al 
and Fig. El) 



4 Large populations with no spatial correlation decay 

In general, for large populations it is difficult to obtain a closed form expression for Ip in terms of the 
variances, correlation coefficients and firing rates. Results are available under different simplifying 
assumptions that make the problem mathematically tractable [H H8"l 143] . In most cases it was 
assumed that correlation coefficients, pij, are independent of the stimulus 6, so that p\- = 0. In 
the following we refer to this as the Stimulus Independent (SI) case, and contrast it to the Stimulus 
Dependent (SD) case. The assumption that we make is that correlations between cell pairs, pij, 
are given by Eq. (|3J), and that stimulus dependence of correlations, Sij(0) takes the product form 
in Eq. flU). 

In this section we let c(0j — <j>j) = 1- Therefore, the correlation structure is completely deter- 
mined by the stimulus. In this case an analytical expression for Q _1 and Ip can be found using the 
Sherman-Morrison Formula |29|, p. 124]. We derive the exact expression for Ip for arbitrary popu- 
lation sizes N, arbitrary response characteristics Vi(9), fi(0), and Si(6), as well as an approximation 
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valid for large populations, in Appendices iBl and ICl 

To give concrete examples of how stimulus dependence of correlations impacts If in large 
populations, in the remainder of the paper we further assume (as in, e.g., [HI H5J HQ [TU] ) , that cell 
responses follow tuning curves that differ only by a phase shift, so that we can write 

W) = f(6 - fa), Vi (9)=v(9-fa), and = s(0 - fa), (10) 

where 9, fa 6 [0,2tt). We take all functions to be periodic. The response, fi{9), is chosen so that 
neuron i responds preferentially (with maximum rate) to stimulus 9 = fa, where fa is fixed. These 
are common assumptions that simplify the analysis considerably [46 \ 05] . Correlations are therefore 
determined by Pij(9) = s(9 — fa)s{9 — (ftj). 

Assuming the neurons sample the stimulus space uniformly and sufficiently densely, we can use 
the continuum limit to approximate Ip. In this case, an arbitrary vector a(0) with components 
a(9 — fa) tends to a function a{9) of the stimulus 9. As we show in Appendix [Cl If can then be 
approximated as the sum of 

(ii) 

where G{9) = (s'(6) + s$s(0)) and 

n( (R\ <fi\\ ~ N \ P m P g W'W ^ / P _fML_ rf J 

D(a(0)A0))»^\J o T^W) d(p ~l T^W) d V h T^(0) # J- (12) 

By symmetry, neither /™ ean , /|? v nor Ip depend on 9 in the large population limit, since the 
response provides equal information about any stimulus. Therefore, we fix 9 = ir in the following, 
and write the firing rates, variances and correlations as functions of the neurons' preferred stimuli, 
(f>. The correlation between two neurons with preferred stimuli <f> and fa will be denoted by p(4>, fa), 
and p(fa) = p((p,fa) = s 2 (fa will be the correlation coefficient between two neurons with equal 
stimulus preference. 

In the remainder of the paper, we make one final assumption: that the functions /, v, and s are 
even (i.e., symmetric around preferred orientations), as in, e.g., |46[ 148] and many other studies. 



Effects of stimulus-dependent correlations on /J? ean : To illustrate how stimulus dependence 
of correlations can influence the information contained in the population response we first consider 
jmean_ E ven w hen correlations are small, this stimulus dependence can have a strong effect via 
correlation shaping. 

Since f{fa) and v(fa) are even, f {fa) / y v(fa) is odd. Therefore, setting a{fa) = f'(fa)/\J v(fa), the 
second term in Eq. (|12p vanishes, and 

N f 27T (f (6)) 2 1 

jmean = ^ / UW^ —dfa (13) 

27r7 v(fa l-s 2 (fa V K ' 

Although /™ oan is the average of the Fisher information [fl] 2 /vi of single neurons, with a weight- 
ing factor, caution needs to be exercised when interpreting this result. Eq. (|13p is the result of 
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cell index cell index cell index 

Figure 2: Examples of different correlation tuning curves and their impact on /™ can for large 
populations. Top panels show the correlation tuning curves, p(cp) = s 2 (<p) for the SD (black) and 
SI (gray) cases along with the (normalized) mean response /(</>) (dashed). Average correlations 
are matched to equal 0.1 in all cases. Insets illustrate the p — / relationship for each choice of the 
correlation tuning. Bottom panels show the integrand of Eq. (|13p for the SD (black) and SI (gray) 
cases, a) p — f follows a concave increasing curve, and p(4>) shows a slightly broader tuning than 
f(4>) in the SD case, resulting in a substantial increase in /™ ean with respect to the SI case (increase 
of ~ 21%). b) p — f is non-monotonic, and p((j)) is bimodal and matches (f'((/))) 2 /v((/)) in the SD 
case. This yields a larger enhancement of /™ ean with respect to the SI case (increase ~ 29%). c) 
Correlations that decrease with rate have a negative impact on /™ ean (decrease of ~ 7% compared 
to the SI case). In all cases /™ ean was computed in the large N limit using Eq. (jlip . Parameters: 
average correlation coefficient s 2 = 0.1 in all cases (larger values, e.g. 0.2, will typically more than 
double the difference in /™ ean between SD and SI cases). In all cases f((p) = 5 + 45a 6 (eft) with 
a{4>) = 1/2(1 — cos(^)), and v(4>) = f((f>) (Poisson). (a) s(4>) =k p + b p a 2 (4>) where k p = 0.135 
and b p = 0.5; (b) s(<f>) = 4r max /(6»)[/ max - /(6<)]//^ ax with r max = 0.65 and / max = 50; (c) s(cp) 
=k p + b p a 2 ((f)) where k p = 0.47 and b p = —0.4. (See Appendix E.) 
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simplifying an expression derived from all pairwise interactions across the population. 

In the SI case, each Sj(0) is independent of the stimulus, and s((f>) is therefore constant across 
the population: s((j)) = s. We focus on comparisons between SI and SD cases matched to 
have the same average correlation coefficient across the population. We therefore assess the ef- 
fects of the stimulus- dependence of correlation, as opposed to the level of correlations. Specif- 
ically, we ensure that the average correlation coefficient across the population in the SD case, 
(47T 2 ) -1 J" 27r Jq 77 s(4>i)s(4>2) dcfti d4>2, equals that in the SI case by setting s = l/(2ir) s((f>)d(j). 
Examples of typical matched correlation matrices, pij, in the SD and SI cases, are shown in the 
right hand column of Fig. [3l 

Panels a) and b) of Fig. [2] illustrate how correlation shaping may increase /™ ean in the SD case 
over the SI case. In each, stimulus-dependence of correlations arises from a different relationship 
between stimulus-induced firing rate and correlation (see insets). In a), p(4>) increases with f(<f>), as 
in [T7] and certain regimes in [HI 1261 f22] . In b), p((j)) first increases with f(4>), and then decreases, 
as in feed-forward networks with refractory effects |45j . Importantly, for both panels a) and b), 
correlations are high between neurons that individually carry most information about the stimulus 
(i.e., between neurons with large values of (f'(cj))) 2 /v(cj))). Therefore, the weighting factor 1/(1 — 
s 2 ((p)) assigns a greater contribution of these more-informative cells to the weighted average in 
Eq. (fT3l) for the SD case, leading to the increase in /™ ean . 

On the other hand, panel c) of Fig. [2] illustrates a case in which correlations decrease with 
firing rates, as observed in [2]. As a result, correlations between the most informative neurons are 
smaller than average, and correlation shaping negatively impacts If- We note that in all panels 
maximum pairwise correlations satisfy /? max < 0.45, within the range typically reported (e.g., 
[23\ [341 149j). Increasing this maximum, without changing the mean correlation, can make these 
correlation shaping effects more pronounced. 

A different way of seeing how /™ ean can be greater in the SD than the SI case is given in Fig. [3k. 
Here, /™ ean and J|? v are computed numerically, and plotted as a function of the population size 
N, for both the SD and SI cases that correspond to the example of correlations increasing with 
rate (Fig. [2k.). Note that /™ oan dominates I|? v over a wide range of N and that the total Fisher 
information, not just I™ ean , is greater in the SD vs. SI case. Moreover, the continuum limit given 
in Eq. (|13p appears valid even for moderate population sizes. 

Care needs to be taken when trying to intuitively understand these population-level effects of 
stimulus-dependent correlations on /™ ean by invoking the case of two neurons studied in Section [3j 
Consider the case of correlations increasing with firing rate (Figs. Ek, Eh.)- As noted in the discussion 
of Eq. ([8]), an increase in correlations between two similarly tuned neurons will typically have a 
negative impact on /™ ean , due to the dominance of the second term of /™ ean in Eq. (JHJ). On the 
other hand, Eq. (|13p shows that increasing correlations between the most informative neurons in a 
large population, regardless of the similarity of their tuning, has a positive impact. The two results 
are not contradictory. Consider the pairwise sum of the two-neuron J™ ean from Eq. ([HJ over all 
neuron pairs in the population. Note that the second term of /™ ean in Eq. ([HJ can be expected to 
be matched with one of equal and opposite sign in such a sum, if the tuning curves are symmetric, 
and correlations depend only on firing rate. Therefore, the typically-dominant second term cancels, 
and it is the first term in Eq. ([8]), always positively impacted by the presence of correlation, that 
remains. Moreover, examination of this first term in Eq. ([HJ does show similarity with Eq. f)13[) : 
in both cases, assigning largest correlations p^j or s(</>, ft) to most-informative neurons will yield 
the greatest total value of /™ ean . 
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Figure 3: /™ ean and /™ v as a function of population size N, for matched SD and SI correlation 
cases and various correlation decay lengthscales. Here, correlation is assumed to increase with 
firing rate, as in Fig. [2^). The coefficients k p and b p defining s(<p) = k p + b p a 2 (cf)) are chosen to 
keep the average correlation coefficients over the population equal to 0.1 (See Appendix |E|). The 
corresponding correlation matrices, pij, for the SD and SI cases are also shown (on-diagonal terms 
are set to in these plots). 
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Fig-HtO shows that this cancellation argument, while not directly applicable, is at least analogous 
to what happens when computing /™ ean for the large population via the complete expression (|lip . 
The sum of the terms f'if'jQj] defines the linear Fisher information, /™ ean = £\ f-f'jQj'j (see 
Eq. ([6])). Under the present symmetry assumptions, the off-diagonal terms cancel, and only the 
diagonal terms contribute to the sum. In Appendix O we show that = [vi(l — in 
agreement with the remaining term in Eq. (|13p . 

These observations are robust to the presence of weak asymmetry in the functions /, v, and 
s. For instance, when the tuning curve f(9) is a sum of a symmetric and small asymmetric 
part, / sy m(0) + e/asym(#)) an examination of Eq. (fT2j) shows that the impact of the asymmetry on 
jmean = £>(^2_ >a (0)) j s D f or d e r 0(eN), while 7^ ean is O(N). However, we show in the next 

section that the large population limit can be changed significantly when — cpj) is not constant. 




<t> * 




Figure 4: Plots of the matrix f-fjQ^j whose double sum determines I|? ean . a) no spatial correlation 
decay, and b) spatial decay with a = 0.25. Top: on-diagonal terms of matrix. Bottom: off-diagonal 
terms (with on-diagonal values set to for ease of visualization) . 

Effects of stimulus-dependent correlations on I£ ov : Having discussed J™ ean , we now turn 
to the impact on I|P V of the stimulus-dependence of correlations. In Appendix[D]we show that this 
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impact is negligible for small to intermediate correlations, and that 

Moreover, as discussed in Section (3[ values of v(4>)2v' ((f)) are typically smaller in magnitude than 
values of ^Jj^ ■ Therefore, for small to intermediate correlations the major contribution of the 
stimulus-dependence of correlations comes from /™ ean rather than I|? v . This agrees with the case 
of two cells (Sec. [3]). Asymptotic estimates of the integrals in /|? v show that this remains true even 
for correlation coefficients close to one. The dominance of /™ ean over J|? v is apparent in Fig. [3^). 
As we show in the next section, however, that this dominance may no longer hold in the presence 
of spatial decay of correlations [Ml H3] . 

Summary of Sec. HJ Stimulus-dependence may shape the structure of correlations so that neurons 
that are most informative about the stimulus presented are most highly correlated. This can lead 
to an increase in overall information. This is possible even when the average correlations across 
the population are low, but not when correlations are fixed, or if all neurons have identical mean 
responses. 



5 Effects of correlation stimulus dependence in the presence of 
spatial decay 

In this section we examine how stimulus-dependent correlations affect Ip in the presence of spatial 
correlation decay. We again assume that correlations and rates are described by Eqs. (J3HH)> but we 
now assume that 



C exp 



a 



The constant a determines the spatial range of correlations, while C was chosen so that the average 
correlation across the population (pij) remains constant as other parameters are varied (for details 
see Appendix |EJ). As an exact expression for the inverse of the covariance matrix is difficult to 
obtain, we study this case numerically, and give an intuitive explanation of the results. 



Effect of correlation shaping on I™ ean : When a = oo, there is no spatial decay, and we 
are in the situation discussed in the previous section: Ip is typically dominated by I™ ean , which 
grows linearly with population size N (Fig. [3h.). However, for finite values of a, /™ ean generally 
saturates with increasing N (Fig. [3b, c). This agrees with earlier findings for stimulus-independent 
correlations [4"3| EH]. 

Additionally, effects of stimulus-dependence in correlations on /™ ean can be reversed for finite 
values of a. For example, assume that Si((p) increases with the firing rate, as in Fig. [2^). When 
a = oo, stimulus-dependence of correlations increases /™ ean (Fig. [3^). However, for finite a, this 
stimulus dependence has a negative impact on /™ ean (Fig. [3j3,c). 

Intuitively, this may be due to spatial correlation decay reducing correlations between neurons 
with differing stimulus preferences. The negative impact of correlations between similarly tuned 
neurons on Ip is no longer balanced by the positive impact on differently tuned neurons. Indeed, 
the stronger the spatial decay of correlations, the more this balance is broken. Therefore, the 
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cancellation arguments presented in the previous section no longer hold - compare Fig. Eb) and c) 
- and it is no longer the case that simply increasing correlations for more-informative neurons will 
increase I™ ean . Instead, correlation structures that increase correlation for similarly vs. differently 
tuned neurons can again be expected to decrease /™ ean . Figure H] shows that this is the precisely 
the effect of the SD vs. SI correlation structures. 

As a second example, assume that correlations decrease, rather than increase, with firing rate, 
as in Fig. [2b. In this case, correlations between similarly tuned, strongly responding neurons are 
decreased. As expected from the arguments above, stimulus-dependent correlations then increase 
jmean Qver j^. g va i ue j n stimulus-independent case. Moreover, absolute levels of If increase 
twofold compared to the analogous case where correlations increase with rate (compare Fig. Efc and 
Fig. Ek). 

However, in all of these cases, note that levels of If are lower in the presence of correlation decay 
for both SD and SI cases. We now mention one way in which this can be mitigated. As illustrated 
in Fig. Eb), we increase the number of areas or subpopulations that respond strongly to a given 
stimulus. The response of each cell still follows a unimodal tuning curve, as above. However, the 
entire population has a number of cells at different spatial locations that share the same stimulus 
preference. Therefore, cells in different subpopulations are only weakly correlated and can be 
thought of as members of different, nearly independent populations. As Fig. (5b) shows, this boosts 
overall levels of /™ ean , while maintaining the benefit of stimulus-dependence in correlations within 
individual subpopulations. 

In sum, the spatial decay of correlations has a strong negative effect on linear Fisher information 
jmean jf correlations depend on stimuli via an increasing relationship with firing rate, this effect 
can be accentuated, with levels of /™ ean decreasing by a further factor of two for SD vs. SI cases. 
However, the opposite effect occurs if correlations decrease with rate: stimulus-dependence can 
then approximately double /™ ean . 

Effect of correlation coding on I F ov : For large populations, Figs. El El show that information 
can be carried predominantly by /|? v , and this dominance is more pronounced as the correlation 
lengthscale a decreases. This agrees with earlier findings [46} 143] . Moreover, we see that the effects 
of stimulus dependence of correlations on J|? v have the same "sign" as those on /™ ean . Specifically, 
when correlations increase with rate, as in Fig. El both /™ ean and I™ v are lower in the SD than in 
the SI cases, for finite values of correlation length a. Also, when correlations decrease with rate, 
as in Fig. El corresponding values of both /™ ean and J|? v are higher for the SD than the SI case. 

The effects of stimulus dependence on the (dominant) /™ v terms can be attributed to corre- 
lation coding. In detail, the contribution of p[j{0) terms to J|? v can be isolated numerically by 
simply computing /|? v twice: once with these terms at the nonzero values expected from stimulus 
dependence, and once after "artificially" setting all of these terms equal to zero. The difference 
is the contribution to If attributable directly to changes in correlation with the stimulus (i.e., 
correlation coding, as opposed to the correlation shaping effects that have been the focus of much 
of the previous discussion). Our calculations (not shown) indicate that almost the entire increase, 
or decrease, of Jp ov in the SD relative to the SI cases is due to this correlation coding. 

Remark 1: More heterogeneous populations of neurons have been shown to yield higher values 
of Fisher information in some cases [441 112j . We modeled such heterogeneity by randomly and 
independently jittering the tuning curves of the neurons, while preserving the expected correlation 
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Figure 5: Examples where the Fisher information is larger in the SD than the SI case, despite 
strong correlation decay a = 0.25. In each case (as in all of Figs. 2-6), s{4>) is set so that the 
average correlation coefficient p across the population is 0.1. a) Correlations decrease with firing 
rate, as in Fig. [2b). b) The response of a population with two subpopulations each tuned to the 
stimulus. In the right panel, as in Fig. [2 (f'((j))) 2 /v{4>) is scaled and represented by the solid line, 
while p(cj)) = s{4>) 2 is represented by the dotted line. The effects of spatial correlation decay are 
not shown. The response of each cell in the population follows a unimodal tuning curve; however, 
there are two sets of cells at different spatial locations, that share the same stimulus preference. 
The left panel shows the effect of this arrangement on the Fisher information. Other parameters 
are as in Fig. [3j 



16 



between pairs. Perturbing the different tuning curves by 10% had a relatively small impact on the 
present results. Specifically, If terms still increased (or decreased) in the same SD vs SI cases. 
Moreover, although /™ ean does not necessarily saturate, for small perturbations I|? v still dominates 
even at large population sizes. 

Remark 2: As discussed in Section [2j it has been observed that correlations between neuronal 
responses decrease with the difference between their preferred stimuli |49t I28| . This effect can 
also follow from stimulus-dependence of correlations: When correlations increase with firing rate, 
two neurons that both respond strongly to similar stimuli will be more correlated than those of 
neurons whose preferences differ. As neurons with similar preferences in stimuli can be expected to 
be physically closer in the cortex, stimulus dependence can result in correlations that decay with 
physical distance [35]. This is quite different from the case where physically distant cells are less 
correlated due to a smaller overlap in their inputs. With stimulus-dependence of correlations, two 
distant cells, one or both of which are responding strongly, may be more correlated than two nearby 
cells that are both responding weakly (see Fig. [3]). 

6 Discussion 

Correlations in the neural response have the potential to both positively and negatively impact 
the ability of a population to carry information about stimuli. Intuitively, correlated fluctuations 
imply a common component in the response noise of different neurons. Similarly tuned neurons 
may provide redundant information, as the common noise component cannot be directly averaged 
away |241 [9j |4"9] . However, it is also possible that noise can be removed by taking differences between 
neural responses pQ. The net effect of correlations on population level information therefore depends 
on the balance among different effects. 

We considered neuronal populations with stimulus-dependent correlations and discussed two 
ways in which such stimulus dependence influences Fisher information. The first, correlation coding, 
refers to the information directly carried by changes in correlation structure in response to stimuli. 
The second, correlation shaping, refers to the impact of stimulus dependence on information carried 
by the mean and variance of neural responses. In different cases, we derived expressions for the 
Fisher information that isolate correlation shaping and correlation coding effects: For cell pairs, 
and small-to-intermediate populations Eqs. ([8]), (J9j) are valid for general correlation structures. For 
correlations with product structure, Pij{0) = Si{6)sj{9), expressions are derived for populations of 
arbitrary size N, with simplifications in the continuum limit N — » oo (Eqs. (|13p and (|14p). 

These expressions allow us to make a number of general observations. For typical firing regimes, 
we find that the effects of correlation shaping dominate over those of correlation coding for pairs 
of neurons or small populations with weak-to-moderate correlations, with most information being 
carried by 7™ ean . Correlation coding only becomes significant for strong correlations. However, for 
large populations the answer is different. Without spatial decay of correlations, correlation shaping 
and J™ ean dominate (cf. [43, |44]) regardless of correlation strength. However, correlation coding 
and I|? v become important in the presence of decay. 

Additionally, for pairs of neurons or small populations with weak correlations, correlated re- 
sponses between similarly tuned neurons typically decrease 7™ ean , while correlations between op- 
positely tuned neurons increase 7™ ean , as has been shown in related settings (cf. [U ETJ EJ 46J). 
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However, for large populations with symmetric and uniformly distributed tuning curves, the sit- 
uation may be quite different. For correlations with product structure and without spatial de- 
cay, Pij(0) = Si(9)sj(9), correlations between the "most-informative" neurons (those with largest 
fi(0)/\/vi(9)) have the greatest impact on /™ ean , regardless of similarity of tuning. Some forms of 
stimulus dependence can increase these correlations, providing a boost to the Fisher information; 
others decrease these correlations and hence the Fisher information. Interestingly, in the pres- 
ence of spatial decay of correlations, these effects of stimulus dependence on Fisher information 
are typically reversed. We note one interpretation: since spatial decay tends to decrease Fisher 
information, the correct stimulus dependence of correlations can counterbalance this effect. 

What biological mechanisms could underly different patterns of stimulus-dependent correla- 
tion? One is the co-tuning of correlation and response rate that has been observed in feed-forward 
networks [17^ 145] . More complex network effects could be behind the decreasing trend of corre- 
lation with rates seen in [2]. Moreover, stimulus-dependent adaptation of correlations has been 
observed in the visual cortex [271 EQl [23] . Our study points to the potentially distinct impacts of 
the mechanisms on population codes. 

Fisher information is only one of the possible metrics that can be used to quantify the impact 
of correlations. However, its close connection with stimulus discriminability [16] . relative ease of 
computation compared to other metrics, and recent use in experimental settings [23], [6] make it 
a good starting point. Future work will extend our study of the impact of correlation stimulus 
dependence to other metrics, such as mutual information, adding to results of [30|. I32j. 

Another important question for future work comes from decoding: how can information encoded 
in correlation changes be read out? For cases in which information is dominated by /™ ean terms, 
a linear readout will suffice; however, when I|? v dominates, as for large populations with distance- 
dependent decay of correlations, nonlinear schemes are required [4"5] . 
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A Fisher information for small populations with small correlations 

The appendices contain a number of exact expressions and approximations of the Fisher information 
for both intermediate and large populations. These results should be useful in the further analysis 
of the impact of correlations in settings similar and distinct from those studied here. 

The approximation in is obtained from the assumption \pij\ <C 1. Defining epij = pij, we 
can write 

Qi,j = SijVi + e(l - <\.j)Pi..j\-<'i'\i- 

Therefore Q is a perturbation of a diagonal matrix R with entries Rij = 5ijVi(x), and the perturba- 
tion eS where Sij = (1 — t>i,j)Pi,j (%) y Vi(x)vj (x) . We can now use the standard matrix perturbation 
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result (see also pi 119] ) 



CT 1 = [R(I + eR _1 S)] 1 = (I + eR-^S) -1 !*.- 1 



i=0 



R 1 



= R 1 - eR _1 SR 1 + eR -1 SR- 1 SR -1 + (e 3 ). 

The equality on the second line holds whenever ||eR _1 S|| < 1 for a norm || 
with itself |19^ Lemma 2.1]. Using (|15j) . we obtain 



Q 



l 



e(l - 5, 



Pi, 



h3 / 



VjVi 




(15) 



which is consistent 



(16) 



Using this equation, the first term in the expression for Ip, f T Q _1 f, can be computed directly, 
to obtain the expression on the first line of ([9]). The second term, Tr[(Q'Q _1 ) 2 ]/2, can be com- 
puted similarly, through a lengthier computation. This computation can be simplified using the 
observations in the next section. This gives Eq. 0, keeping terms up to second order. 

The convergence of the sum on the second line of (|15p is not guaranteed if ||eR _1 S|| > 1. This 
implies that for fixed e, the approximation (I16D will break down for sufficiently large N (typically 
about when N > 1/e). 



B General expression for Ip in the product case 

In this section we use the Sherman-Morrison Formula [29\ p. 124] to derive a general expression 
for the Fisher information in the product case. Let 



A' 



1 (i - «?) 



(17) 



Then 



if i = j 



if i / j. 



(18) 



Using this equation we can obtain a compact expression for Ip. The term resulting from changes 
in the mean number of spikes as the stimulus varies is given directly from definition (|7|) as 



v 



I— {x) = f>f>Q-j. 



(19) 



The contribution to If due to changes in the covariance, given by I|? v =Tr[(Q'Q ) ]/2, can 



19 



be expressed compactly by introducing 



Ri 



d_ 

dx 



Ins,; 



and Zi 



dx 



hx{s i - s /v., 



s A + l v A. 

Si 2 Vi 



(20) 



Note that when pij have the form given in Eq. ([3]), c(0j — 4>j) = 1, and the stimulus dependence of 
correlations, Sij(6) takes the product form in Eq. (J4j) If , we can write 



Q i j — {Zi + Zj — 25ijRi)Qi, 



3i 



where Z, and Ri are defined in (|2Up . Following this observation, we can follow the computations 
in Appendix A] , to obtain 



Tr[(Q'Q 



N 



N 



N 



N 



5^ QkjZkZiQ^l — & y y Qk,kZkRkQ k\ + ^ y~] Qk,kQi,iRkRiQk}Qi,k- 



k=l k,l=l k=l 

Observing that Q _1 is self-adjoint, we obtain 

N 



k,l=l 



If v = ^(Ztf [l + Q'lviil - s?)] + ^ ZiZjSiSj^vvJQrj 

^ 2 W 

+ 2^i?^ ^/WjQi} -A^ZiRiViQrl. 



(21) 



i=l 



Therefore, 7p is the sum of (|19p and (|2ip . 

The contribution to /p due to only changes in the variances can be obtained from Equation ([2T 
by setting i?, = and replacing Z, by v' i /{2vi), so that 



i=l x ij v J 

The contribution due to correlation stimulus dependence is therefore 



(22) 



7-corr rcov rvar 

J F — 1 F ~ 1 F ■ 



C Asymptotic results 

The expression for Ip derived in Appendix [B] can be simplified considerably for large cell popu- 
lations. If N is large and < e < s, < 1 — 8 for some e, 5 > 0, then 5 = O(N), where S is 
defined in (fT7|) . The assumptions on Sj are not essential, but make the derivation of the asymptotic 
expressions easier. 
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Keeping only the leading order terms in (|18p we can write 

1 



«((!--?) 



if i = j 



if i / j. 



(23) 



To obtain the asymptotic value of Ip given in (|26p from Eqs. (|19p and ()2ip . first note that 
5 = O(N). Therefore, for large JV, 



N 2 N 



Using this observation together with the asymptotic value of Q i I given in (|23p . the first, and last 
two sums on the right hand side of (|2ip behave asymptotically as 



N 



N 



N 



N 



i=i 



»=i 



i=i 



Ri 



By a slight abuse of notation define the weighted average of the entries in the vector a over the 
population as 



JV ^ 1 - s 2 \ 1 



and let 



D(a, s) = f JV 



1 - s 2 / \ 1 - s 2 
Then the observations above can be combined with 



> 2 / 2 
as \ , / s 
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N 9 



W „2 
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1 - s 2 



E »i 1 I \ ^ "j I _ / a i s i 
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(24) 
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(25) 



l-s 2 



def 



,l-s 2 /\l-s 2 / \l-s 2 
D(a,s). 

applied to the term /™ ean and the second sum on the right hand side of (|2ip . gives 

2 



jmcan^ 



D(—,s), and If v (x)~2V 



N , , , 

V; SS 



2vi 1 — sf 



(26) 



where Gi = ^ln(sjy^) = s^/sj + As before, J£? ean corresponds to the linear Fisher 

information. 
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Figure 6: Values of /™ can and I™ v from: i) approximations for small p, valid for intermediate 
population sizes N, given by Eq. Q it) the "exact" value obtained by numerically inverting the 
correlation matrix Q, and using Eqs. (J6H2]), Hi) the large N approximation given by Eq. 1126)1 . and 
iv) the continuum limit given by Eq. (TTTHT2J) . Here, f(<p) = 5 + 45a(</>) with a((f>) = 1/2(1 + cos ((/>)), 
and v(4>) = f{4>) (as for Poisson variability). Additionally, s(4>) = 0.2 + 0.5a((p) . Other parameter 
choices give similar results (not shown). 

The Cauchy inequality can be applied directly to show that 



ar 



>0, 



so that D(-,s) is always positive. 

Figure [6] shows that the approximations, together with the continuum limit expressions found 
in the main text, are valid to high accuracy over broad ranges of N. 



D Impact of pure correlation stimulus dependence on 7|P V . 

We show that the impact of stimulus dependence of correlations on I|? v is relatively small compared 
to the impact on /™ ean in the situation discussed in Section SI By invoking the symmetry of the 
tuning curves again 



1 



dig.,.) - DiA<» + ^sm.sm) ~ l[' (m + ^s Wj 1 _ 



(27) 



where s'(6) is typically much smaller than s(6)v'(6)/v(9). The term D(Gs,s) appearing in 7|? v is 
therefore of second order in s(9) and hence negligible compared to I|? ean . For typical parameters, 
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the difference is greater than an order of magnitude. 

The last term in the Fisher information comes from the sum in 7|? v given by Eq. (|26p . In the 
continuum limit this term is approximately 

N f 27T ( v{$) s'(<f>)s((f>)\ 2 JA N f 2n f d r , a^yA 2 ,, 

For the type of stimulus dependence that we assume 2 ^^) an d have opposite signs. 

For small correlations, the first term will dominate and stimulus dependence of correlations will 
decrease this entry in I|? v . When correlations are not perfect (near 1) the term I™ 1 is typically 
much smaller than /™ ean . 

E Details of the numerical implementations 

Numerical values of Fisher Information in Figs. [3] and [5] were found by directly inverting the cor- 
relation matrices Q and performing the required matrix multiplications in MATLAB. The authors 
are happy to provide these codes upon request. 

The procedure is as follows: We first fix the average value of correlations, (pij), among all 
neurons in the population (the value (pij) =0.1 was used for all figures in this paper). Next, we 
define correlation matrices consistent with this value of (pij), for two cases, Stimulus Dependent 
(SD) and Stimulus Independent (SI) (see main text). We first define Qij via Eqn. (j2J), assuming 
that the Pij(6) are given by ©. Here, for Figs. [3] and [5l we used s(9) =k p + b p a 2 (8), where 
a(6) = 1/2(1 + cos(0)) and k p and b p are constants chosen as follows: (i) the average correlation 
(pij) = 0.1, and (ii) the ratio of largest so smallest pairwise correlations, (k p + b p ) 2 /b 2 , should be 
R = 10 for the SD case and R = (i.e., b p = 0) for the SI case. 

To study the affects of heterogeneity, as a final step we jitter the tuning curves for s and v by 
±20%. 
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